Development of Hybrid System for Minimally Invasive Surgeries.

Minimally Invasive Surgery(MIS), although advantageous compared to open cavity surgery in many aspects, is not widely accepted by surgeons for clinical application. This is due to cost and required complex training of the currently practiced technology. The main challenges in reducing cost and amount of training is to have an accurate inner body navigation advisory system along with the use of a flexible robot arm to safely perform such surgery. As a first step in making minimally invasive surgery affordable and easy to use, quality images inside the patient body as well as accurate position of the surgical tool should be provided in real time. An array of asynchronous sensors is required to successfully and safely perform real time inner body navigation. Furthermore, it is necessary to have flexible surgical arm capable of following curved paths to avoid damaging patient organs when moving the surgical tools inside the body. In this work, three asynchronous sensors are calibrated and information were fused together in real-time spatially and temporally in such a way that the fused information will be useful to perform successful MIS.  Furthermore, a complete design of the flexible surgical arm system is proposed. It is believed that the result of this work will have practical impacts on making MIS more affordable, and flexible surgical arms more accurate and user friendly than existing counterparts

Design, Integration, and Field Evaluation of a Robotic Blossom Thinning System for Tree Fruit Crops

The US apple industry relies heavily on semi-skilled manual labor force for essential field operations such as training, pruning, blossom and green fruit thinning, and harvesting. Blossom thinning is one of the crucial crop load management practices to achieve desired crop load, fruit quality, and return bloom. While several techniques such as chemical, and mechanical thinning are available for large-scale blossom thinning such approaches often yield unpredictable thinning results and may cause damage the canopy, spurs, and leaf tissue. Hence, growers still depend on laborious, labor intensive and expensive manual hand blossom thinning for desired thinning outcomes. This research presents a robotic solution for blossom thinning in apple orchards using a computer vision system with artificial intelligence, a six degrees of freedom robotic manipulator, and an electrically actuated miniature end-effector for robotic blossom thinning. The integrated robotic system was evaluated in a commercial apple orchard which showed promising results for targeted and selective blossom thinning. Two thinning approaches, center and boundary thinning, were investigated to evaluate the system ability to remove varying proportion of flowers from apple flower clusters. During boundary thinning the end effector was actuated around the cluster boundary while center thinning involved end-effector actuation only at the cluster centroid for a fixed duration of 2 seconds. The boundary thinning approach thinned 67.2% of flowers from the targeted clusters with a cycle time of 9.0 seconds per cluster, whereas center thinning approach thinned 59.4% of flowers with a cycle time of 7.2 seconds per cluster. When commercially adopted, the proposed system could help address problems faced by apple growers with current hand, chemical, and mechanical blossom thinning approaches

Machine Vision System for Early-stage Apple Flowers and Flower Clusters Detection for Precision Thinning and Pollination

Early-stage identification of fruit flowers that are in both opened and unopened condition in an orchard environment is significant information to perform crop load management operations such as flower thinning and pollination using automated and robotic platforms. These operations are important in tree-fruit agriculture to enhance fruit quality, manage crop load, and enhance the overall profit. The recent development in agricultural automation suggests that this can be done using robotics which includes machine vision technology. In this article, we proposed a vision system that detects early-stage flowers in an unstructured orchard environment using YOLOv5 object detection algorithm. For the robotics implementation, the position of a cluster of the flower blossom is important to navigate the robot and the end effector. The centroid of individual flowers (both open and unopen) was identified and associated with flower clusters via K-means clustering. The accuracy of the opened and unopened flower detection is achieved up to mAP of 81.9% in commercial orchard images

Minimally Invasive Surgery Using Heterogeneous and Asynchronous Sensors-Part I: Offline Sensors Calibration

Minimally invasive surgery (MIS) is not currently widely used by surgeons due to its cost, and complex training requirement of   the existing technology. As a first step toward making MIS a more accessible technology to use is to provide the surgeon with quality images inside the patient as well as the surgical tool location automatically in real time in a common reference frame.  The objective of this paper is to build a platform to accomplish this goal. A set of three heterogeneous asynchronous sensors is proposed to help the surgeon navigate surgical tools inside the human body. The proposed system consists of a Laser Range Scanner(LRS) to emulate the pre-op CT/MRI, an Electromagnetic Tracking System(EMTS), and a small size Camera.  This set of sensors provides all the necessary information needed for MIS navigation. The sensors have different data rate, reference frames, and independent clocks. A prerequisite for successful navigation   is to represent all the sensors data in a common reference frame.  The focus of this paper is on off line calibration of the three sensors, i.e. before the surgical device is inserted in the human body.  This is a prerequisite for real time navigation inside the human body. The proposed off line calibration technique was tested using experimental laboratory data. The result of calibration was promising with an average error of 0.1081mm and 0.0872mm along the x and y directions, respectively, in the 2D camera image

An autonomous robot for pruning modern, planar fruit trees

Dormant pruning of fruit trees is an important task for maintaining tree health and ensuring high-quality fruit. Due to decreasing labor availability, pruning is a prime candidate for robotic automation. However, pruning also represents a uniquely difficult problem for robots, requiring robust systems for perception, pruning point determination, and manipulation that must operate under variable lighting conditions and in complex, highly unstructured environments. In this paper, we introduce a system for pruning sweet cherry trees (in a planar tree architecture called an upright fruiting offshoot configuration) that integrates various subsystems from our previous work on perception and manipulation. The resulting system is capable of operating completely autonomously and requires minimal control of the environment. We validate the performance of our system through field trials in a sweet cherry orchard, ultimately achieving a cutting success rate of 58%. Though not fully robust and requiring improvements in throughput, our system is the first to operate on fruit trees and represents a useful base platform to be improved in the future